The present disclosure relates to automated question answering (QA) systems and methods, and more specifically, to such QA methods and systems that score evidence sources used to support candidate answers.
Most existing factoid question answering systems adopt search strategies and scoring methods with the assumption that a short passage exists in the reference corpus which contains sufficient information to answer each question. This assumption largely holds true for short and focused factoid questions such as “When did Hawaii become a state?’” and “What strait separates North America from Asia?’” However, some more complex factoid questions contain facts encompassing multiple facets of the answer, which often cannot be found together in a short text passage. Consider the following examples:
(1) Born in 1956, this Swedish tennis player won 6 French Opens & 5 straight Wimbledons (A: Björn Borg); and
(2) A small- and medium-vessel vasculitis characterized by allergic rhinitis, asthma, and peripheral eosinophilia (A: Churg-Strauss Syndrome).
In both examples, information presented in the question can reasonably be expected to be in documents that describe the respective answer entities. However, it is quite unlikely that all the information will be present in one or two adjacent sentences in the document. More specifically, in example (1), birth year and nationality information is in the basic biographic section of documents about Björn Borg, while statistics about his tennis record can generally be found in a section about Borg's career. Similarly for example (2), being a small/medium-vessel vasculitis can generally be found in an initial definitional sentence about Churg-Strauss Syndrome, while the rest of the question naturally falls under a section that describes typical symptoms of the syndrome. As a result, a typical passage retrieved from most reference corpus would cover only a portion of the facts given in the question.
These multi-faceted factoid questions present a challenge for existing question answering systems which make the aforementioned assumption. Consider the following short passages relevant to the question in example (2):
(2.1a) Churg-Strauss syndrome is a medium and small vessel autoimmune vasculitis, leading to necrosis.
(2.1b) The Churg-Strauss syndrome is a multisystem disorder characterized by chronic rhinosinusitis, asthma, and prominent peripheral blood eosinophilia.
(2.2a) Wegener's granulomatosis is an incurable form of vasculitis that affects small and medium-size blood vessels.
(2.2b) Wegener granulomatosis is a rare multisystem autoimmune disease of unknown etiology. Its hallmark features include necrotizing granulomatous inflammation and pauci-immune vasculitis in small- and medium-sized blood vessels.
Existing systems evaluate each passage separately against the question would view each passage as having a similar degree of support for either Churg-Strauss syndrome or Wegener's granulomatosis as the answer to the question. However, these systems lose sight that even though each passage covers half of the facts in the question, (2.1a) and (2.1b) cover disjoint subsets of the facts, while (2.2a) and (2.2b) address the same set of facts.